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DISTRIBUTED PROCESSING OF HIGH LEVEL PROTOCOLS, SUCH AS 
REAL TIME TRANSPORT PROTOCOLS, IN A NETWORK ACCESS 

SERVER 

5 

CROSS-REFERENCE TO RELATED APPLICATION 

This is a continuation in part of the patent application of Daniel L. Schoo, et 
al.. Serial No. 08/ 486,591 filed June 7, 1995, pending, the contents of which are 
0 incorporated by reference herein. 

BACKGROUND OF THE INVENTION 
A. Field of the Invention 

s 

Tliis invention relates to the field of data communication and apparatus and 

methods for receiving and transmitting data, video and/or audio communications 

through a communications access system, such as an integrated network access server. 

More particularly, the invention relates to apparatus and methods for distributing the 

processing of higher level network protocols for a single communication session in 

such a system among multiple computing platforms. 

Examples of such higher level network protocols which are subject to 

distributed processing described herein are the Point-To-Point Protocol (PPP), the 

Serial Line Interface Protocol (SLIP), and the Real-time Transport Protocol (RTP). 

However, the principles and methods of the invention are applicable to other types of 

higher lev^el protocols. 

As described in detail below, the processing of the protocols is distributed in 

the preferred embodiments such that one computing platform, such as the computing 

platform at a local or wide area network gateway, performs part of the processing of 

the protocol, while another computing platform, e.g., the computing platform in a 

digital signal processor (DSP) or modem module, performs the remainder of the 

1 
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protocol processing. In this manner, the throughput and efficiency of the call 
processing of the network access server is substantially increased. 
B- Description of Related Art 

The methods disclosed herein can be performed by an element of 
5 communications equipment referred to herein as a "network access server." The 
network access server is a device that is capable of receiving a plurality of 
simultaneous incoming calls from the Public Switched Telephone Network (PSTN) 
and routing them to a packet switched computer network for transmission to a host 
compuier system, or telephone or other device connected to the computer network. 
10 The network access server is also capable of handling multiple simultaneous calls 
from ihc computer network and directing them onto a communications link in the 
PSTN for transmission to the remote user. 

The patent to Dale M. Walsh et al., U.S. No. 5,525,595, which is fully 
incorporated by reference herein, describes an integrated network access server 
15 suitable for use in the present invention. Such a device has been commercialized 
widely by 3Com Corporation (previously U.S. Robotics Corp.) under the trade 
designation Total Control ™ Enterprise Network Hub. Network access servers 
similar in functionality, architecture and design are available from other companies, 
including Ascend Communications, Livingston Enterprises, Multitech, and others. 
20 The invention is suitable for implementation in network access servers from the above 
companies, and other similar devices. 

In order to facilitate such communication, industry and international standards 

bodies have established sets of functional requirements, conventions or rules that 

govern the transmission of data over both telephone and packet switched computer 

25 networks. These functional requirements or rules are known in the art as ''protocols.'' 

2 
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The implementation of protocols is necessary in order to bring order, and 
standardization, to the communications field and allow equipment of diverse 
manufacturers to be interoperable. Some protocols are considered low level 
transmission media related modulation protocols, such as modulation schemes 
5 implemented in a modem, for example V.34, V.22 bis, etc. Other protocols are 
considered higher level, and relate to such features as error control, transmission 
control protocols and network level routing and encapsulation of data. The 
requirements of these latter protocols are typically prepared as a "Request For 
Comment" document, circulated among the industry, and eventually adopted by the 
0 standards bodies. The present invention is concerned with the distributed processing 
of these higher level network control protocols. Examples of such protocols are the 
Point-to-Point Protocol (PPP), the Serial Line Interface Protocol (SLIP), and the Real- 
time Transport Protocol (RTP). 

In order to process communications between the computers on the local area 
network and the remote computers connected via the telephone system, the processing 
of the higher level protocols must be performed. In the prior art, a single platform at 
the network interface has been used to perform the higher level processing. It has been 
discovered that this results in inefficiencies and loss of throughput, particularly when 
the maximum call capacity in the network access server in either the inbound or 
outbound direction is approached. 

In our distributed processing invention, a dramatic increase in the efficiency of 
the call routing process can be achieved, thereby maximizing call throughput and 
minimizing the overall call connect time. This result is achieved by distributing the 
computationally intensive protocol processing of higher level network protocols (such 
as Point-to-Point Protocol ( PPP) processing or RTP) among multiple computing 

3 
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platforms such as the modems of the network access server assigned to each of the 
sessions. As another example, processing of the protocols may be distributed by 
performing some of the processing in one platform at the gatewaiy and performing 
cc^mpulationally intensive protocol processing in the modems. For example, RTP 
[^roiocols associated with audio and video conferencing may be distributed such that 
some processing of RTP is performed in the gateway platform, for example RTCP, 
II*. I T>P and central coordination of RTP streams, while the modem or DSP platform 
pcriomis RTP jitter buffering and encapsulation of audio frames with RTP header 
ilata In addition, the modem or DSP platform perform lower level protocol 
prtvcssinu. including all audio processing, transcoding, DTMF (dual tone 
ir.uititrcqucncy lone) generation, and echo cancellation. 

\ anous types of communication devices are placed at the interface between a 
fniHicin jiui a computer network, such as routers, terminal servers, and modules 
s*Miic!:r:K-s referred to as "gateway cards." These devices implement software 
I^ro-rjrns thai control the inflow and outflow of calls between the modems and the 
iK-iutuk These devices are referred to herein generially as ''network interface" 
U . N ( >ric layer of the software hierarchy that is run in these devices is known in 
the ar: as an "application layer". This document makes occasional reference to the 
terms ' applications", "application layer" and "application software layer." As used 
herein, ii:ese temis means a communication control and management software layer 
abo\c the protocol stacks in a communication device, the device typically placed at 
the uateu ay (or interface) between a modem and a computer network. 

1 o belter illustrate some of the advantages of the invention, one embodiment 
of ihc iineniion is described herein in which the modem or DSP platform in the 
neiu ork access ser\'er perform asynchronous High-level Data Link Control (HDLC) 
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framing of Point-to-Point Protocol (PPP). The protocol could be synchronous HDLC 
framing of PPP in other implementations. The modem performs the flag sequence, 
data transparency, and Frame Check Sequence (FCS) on each PPP frame. The second 
protocol processing performed in the modem or DSP platform is Serial Line Internet 
5 Protocol (SLIP). 

In the prior art, when an application software routine at the network access 
serv er gateway creates a PPP (or SLIP) frame, it checks each byte, looking for a byte 

I ihal is a control character. If the application finds a control character, a PPP Escape 

character (or SLIP Escape character) is stuffed into the data stream. Then, the original 
!<• control character is translated to a transparent character and stuffed into the data 
stream. This usually requires two buffers, because extra characters are added. For 
IM^P frames, while the application is looking at each byte of the frame, it must also 

^ calculate the FCS. When the application receives a PPP (or SLIP) frame, it must do 

the reverse of the above process. In some network access servers, such as the Total 

i \> Control network access server of 3Com Corporation, a large number of modems 

may be active at any one time. This means that the gateway computing platform in 
the netw ork access server would be doing this process for each of the modems if the 

I prior art technique was used. This results in a extremely heavy processing load on one 

j computing platform, and introduces latencies and delays in the call routing process. 

20 These effects combine to significantly reduce call throughput, particularly where a 
large volume of calls are simultaneously received or transmitted through the network 
access server. This call latency and delay is substantially reduced in accordance with 
the methods and system described herein. Similar improvements are expected in an 
situation wherein distributed processing of other higher level protocols is performed, 

: 25 such as RTP, as set forth below. 

5 
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SUMMARY OF THE INVENTION 

; The present invention provides a method for routing incoming or outgoing 

calls between a computer network and the PSTN. For incoming calls, the method 
5 comprises the steps of receiving the incoming calls and directing the calls to a plurality 
of modems, distributing the processing of protocols for the incoming calls among 
multiple computing platforms in the modems and processing the protocols for the 

1 incoming calls, and subsequently routing the incoming calls to the computer system 

over a computer network via a network interface or gateway. For outgoing calls, the 
10 calls are directed from the computer network to a plurality of modems in the network 
access ser\'er. The protocol processing is distributed among a plurality of computing 
platforms in the modems, and the calls are sent out to a communications link such as a 

^ T I telephone line. 

i In one preferred embodiment of the invention, the method is performed in a 

15 network access server having a plurality of modems for receiving a plurality of 
incoming calls or modulating a plurality of calls onto a communications link. The 
computing platforms comprise the data processing structures in each modem. 

I One of the embodiments described below provides for the distributed 

processing of the RTF protocol associated with real-time video or audio traffic. Part 
20 of the processing is performed in the computing platform in the network interface and 
some of the processing is performed at one or more computing platforms in the 
modem. Distributed processing of RTF in this fashion provides a number of distinct 
advantages, both in terms of scaling, efficient use of computing resources, and delay 
reduction. 

6 
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First, the technique deals with talk spurts from the packet switched computer 
network side (such as a local area network or LAN) to be handled in an efficient 
manner. If the entire RTP protocol was implemented on the network interface 
platform, it would make the development of a solution for a phenomenon known as 
5 "talk spurt", i.e., a burst of audio data, more difficult and cause scaling problems in a 
high density network access server. One gateway platform can support many different 
high density and general purpose DSP cards (for example, cards dedicated at least in 
I part to modem functions), each performing a number of functions. Using the 

techniques described herein, hundreds of calls can be processed simultaneously in a 
10 single network access server. 

Second, it uses resources more efficiently, particularly in an environment in 
which a network operating system is run on the gateway computing platform, such as 
Windows NT^^' or UNIX-based operating systems. Distributed RTP processing is 
designed to allow the gateway application to transfer the RTP packet through the 
15 gateway to the LAN in one kernel mode operation. Conversely, since the operating 
system running on the general purpose DSP or modem platform is particularly suitable 
for real time traffic processing, it can effectively handle the RTP end-point 
^ functionality. 

Third, the method provides for reduced end to end delay. The invention not 
; 20 only reduces the overall delay in the system, it hides the variation in delay inherent in 

packet switched computer network from the end user. Such advantages are 
particularly important in real time voice or video applications. The method provides 
for dynamic jitter buffering in the modem and/or general purpose DSP cards,- which 
adapt to delay due to internal processing within the network access server as well as 

25 delay in the computer network traffic. The term "jitter", as used herein, refers to the 
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variation in delay in both the packet switched network and within the network access 
server. Dynamic jitter control can be accomplished by increasing or decreasing the 
amount of data that is being buffered within the system. This buffering mechanism is 
able to account for, as accurately as possible, the actual jitter or variation in delay 
5 characteristics of the packet switched computer network while shrinking the overall 
delay within the real time processing provided by the network access server. 
Moreover, if all RTP processing were done at the gateway platform, a simple jitter 

i buffer would still be required at the PSTN interface due to the jitter added by the RTP 

processing itself inside the system. The basic need is to take an asynchronous data 
sicani (from the packet switched network) and convert it to a synchronous data stream 

: lor the PSTN carrier system without noticeable impact to the user. By implementing a 

inter buffering mechanism as a close as possible to the synchronous PSTN interface 

i (i.e., in a modem or general purpose DSP computing platform), the need for a second 

buffer, as explained above, is removed along with the delay that would be attributed to 
! this data storage within the system. 

\ These and many other advantages and features of the invention will become 

more apparent from the following detailed description of presently preferred 

^ embodiments of the invention. 

i 

20 BRIEF DESCRIPTION OF THE DRAWINGS 

Presently preferred embodiments of the invention will be described in 
conjunction with the drawings, in which like reference numerals refer to like elements 
in the various views, and in which: 

FIG. lA is a block diagram illustrating one form in which the invention may 
^ 25 be implemented; 

' 8 
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FIG. IB is a block diagram illustrating the implementation of the routing 
function of FIG. 1 A in the operating system software of a host computer; 

FIG. 2 is an illustration of the overall communications system in which an 
alternative form of the invention is implemented, illustrating the relationship between 
5 various call originators, a telephone network, a network access server, and a host 
computer system linked to the network access server via a network; 

FIG. 3 is a schematic block diagram of a network access server; 

FIG. 4 is a flow chart of the distributed processing procedure; 

FIG. 5 is a detailed flow chart of the PPP routine of FIG. 4; 
10 FIG. 6 is a graph of the upload throughput through the network access server 

of FIG. 3 using prior art processing in a single platform at the network gateway for 
performing PPP processing; 

FIG. 7 is a graph of the upload throughput through the network access server 
of FIG. 3 when the distributed processing technique according to the teaching of the 
15 present invention is used; 

FIG. 8 is a graph of the download throughput through the network access 
server of FIG. 3 when the PPP processing is performed according to the prior art 
technique at a single platform at the network gateway; 

FIG. 9 is a graph of the download throughput through the network access 
20 server of FIG. 3 when the distributed processing technique according to the teachings 
of the present invention is used; 

FIG. 10 is a block diagram of a high density network access server which may 
be used to perform the distributed processing of the present invention; 

FIG. 11 is a diagram illustrating the distribution of functionality performed by 
25 the computing platform in the high density modems cards in the embodiment of FIG. 
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10, and the functionality provided by the computing platform in the gateway or 
EdgeServer ™ card; 

FIG. 12 is a high level diagram of the distributed processing architecture for 
the RTP protocol in the network access server embodiment of FIG. 10; 
5 FIG. 13 is a more detailed diagram of the distributed processing of the RTP 

protocol in the network access server of FIG. 10, showing the processing performed at 
the EdgeServer platform and the processing performed in the modem platform; 

FIG. 14 is a illustration of the layers of the S-Bus driver interface controlling 
the transmission of messages and data between the EdgeServer card and the modem 
10 card; 

FIG. 15 is an overall block diagram of the hardware for the high density 
modem cards of FIG. 10; 

FIG. 16 is a diagram of the software architecture for the high density modems; 

FIG. 17 is a block diagram of the control processing architecture for the high 
15 density modems, showing the division of processing between the RISC chips and the 
DSPs; 

FIG. 18 is a block diagram of the data processing architecture for the high 
density modems, showing the division of processing between the RISC chips and the 
DSPs, showing the processing of the RTPAJDP and IP headers by the RISC chips for 
20 inbound traffic, and jitter processing, and RTP/UDP/IP processing by the DSPs for 
outbound traffic; 

FIG. 19A is an illustration of one possible configuration of the DSPs for the 
high density modem card, showing the type of transcoding that may be performed by 
the DSPs in an Internet telephony application based on RTP; and 
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FIG. 19B is an illustration of a more versatile configuration of the DSPs for 
the high density modem card, showing the type of tasks that may be performed by the 
DSPs, in an embodiment in which the network access server provides for both remote 
computer network access and Internet telephony types of functionality. 
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DETAILED DESCRIPTION OF THE PREFERRED AND 
ALTERNATIVE EMBODIMENTS OF THE INVENTION 

5 

OVERVIEW^ AND DISTRIBUTED PPP AND SLIP PROCESSING 
EMBODIMENTS 

Referring to FIG. 1 A, the invention may be implemented in a communications 

10 system in which a call originates from a computer, such as a PC 20, which sends data 
via a Data Communications Equipment (DCE) (such as modem M) onto telephone 
network 50 or other communications link to a receiving DCE 10, such as a modem. 
The call originating data terminal 20 has communication software that uses a 
communications protocol, such as PPP or SLIP. The DCE 10 demodulates the call 

15 from the personal computer 20 and passes it over a transmission means 11, for 
example an RS 232 cable or packet bus, to a router or terminal server 12, The router 
or terminal server 12 passes the call onto a network or host computer system for 
example a personal computer (not shown in FIG. lA). Up to n DCE's 10 may be 
provided, depending on the amount of expected incoming traffic. Or, for some 

20 applications, only a single DCE or modem 10 may be required. The DCE 10 and 
router 12 functions may be implemented in physically distinct hardware, or combined 
into a single piece of equipment such as in the case of the network access server 
discussed in conjunction with FIG. 2. 

In a preferred form of the present invention, the byte to byte transparency 

25 translation processing required by the PPP (or SLIP) protocol is distributed to the 
DCE 1 0 processor, rather than being performed in a computing platform in the router 
or terminal server 12 as in the prior art. This frees the application processor (such as 
the computing platform at router or terminal server 12) from this time consuming task. 
The modem 1 0 processor already deals with the data on a byte by byte process as it 

12 
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passes data to or from the modem's data pump. The additional task of data 
transparency and FCS calculating is not as burdensome as it is for a multi-session 
application, and hence the modem processor is a preferred computing platform in 
which to implement the invention. 
5 In our preferred form of the invention, the application software processor at the 

network interface communicates with the modems 10 via a packet bus, but the 
communication could be by other equivalent means, such as an RS 232 cable. Two 
types of messages are passed between them, such as configure requests and responses, 
and data. Other message types are not applicable to this implementation. All 

10 messages have a status field and a length field. There is exactly one PPP or SLIP 
frame transmitted or received per each packet bus data message. The length field 
holds the exact number of untranslated data bytes. This is how the modem knows 
where to insert the flag sequence on outgoing frames. 

The basic design of the PPP/SLIP modem 10 is for the gateway application 

15 software at the router/terminal server 12 to issue a special command to the modem 10 
that puts the modem into PPP mode, SLIP mode, or neither. Other configuration 
commands that can be issued include: local Async-Control Character-Map, remote 
Async-Control-Character-Map local Delete character translation (i.e. ASCII 7F hex), 
remote Delete character translation, maximum fi-ame size, frame timeout, inter- 

20 character timeout, and FCS type (e.g. CCITT 16 bit CRC). The modem 10 informs 

the application software when it is successfully configured. 

When the modem 10 receives a data message fi-om the applicafion layer for 

transmission to the remote PC 20, the modem 10 knows how much data is in the 

message from the length field. The modem then creates a PPP or SLIP frame by 

25 transmitting a frame end character, transmitting the data (translating as required), 

13 
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calculating and transmitting the FCS if it is a PPP frame, then transmitting another 
frame end character. Note that the PPP address and control fields should be 
prepended to the PPP data when passed to the modem because the modem does not 
interpret any data. 

> When the modem 10 receives a PPP or SLIP frame from the remote location, 

the modem 1 0 searches for start of the frame character (which is also the frame end 
character). This character is discarded. The modem then examines each incoming 

I byic of data, translating transparent data as required and keeping a current FCS if it is 

a PPP frame, while looking for the traihng frame end character. The traihng frame 
10 end characier is also discarded. If the modem is configured for PPP. the current FCS 
\ alue is confirmed to be valid and the last two (or four) data characters, which are the 
FCS characters, are also discarded (i.e., the length is decremented). This frame of raw 

: data (prepended by the address and control fields if PPP) is immediately passed to the 

application layer in the gateway (such as the router/terminal server 12) via a packet 
15 bus or an equivalent means. Status values indicate OK, invalid FCS, frame too large, 
intcr-char timeout, frame timeout, and parity error (frame aborts are discarded and the 
application layer is not informed). 

^ Note that, using the techniques of the present invention, the application 

computing platform at the network interface never sees any transparent data. It deals 

20 with the actual information in the PPP or SLIP frame. Also note that the modem does 

not interpret the contents of any PPP or SLIP frame. It only does the most basic 

encapsulation of a datagram over a serial link, i.e., byte by byte transparency 

translation and FCS. The PPP address and control field and any negotiation, such as 

LCP or IPCP, are handled at the application software layer. Accordingly, by using the 

25 techniques of the present invention, data through-put in both the uploading and 
] 14 
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downloading directions through the modem 10 and router/terminal server 12 is greatly 
improved. This is a result of both reduced processing requirements at the gateway 
computing platform, and a lower overall latency of the system. 

Referring again to FIG. lA, it will be understood that other analogous and 
functionally equivalent DCE's may be used in accordance with the invention besides 
ilic modem 10. For example, other computing platforms, such as a those in a DSU 
(data synchronizer unit) and CSU (circuit switching unit) may be used in conjunction 
uiih a digital data network 50. An ISDN (integrated services digital network) 
icnninai adapter may also be used where the calls come in via ISDN lines. 

Rct'crrmg to FIG. IB, it will be appreciated that the physical location of the 
router or terminal server 12 is not particularly important. For example, the router 
lunctuni may exist as a software feature in an operating system 14 for a personal 
eoniput^T, lor example, the various versions of the Windows® program of Microsoft 
CtirporatiiMi in this example, the protocol processing is performed in the modem 10. 
15 U\ \ inuc of ihc distribution of the protocol processing to the computing platform in 
tiic modem ]<♦. rather than in the CPU for the computer, the computational load on the 
C 1*1 aiu) the latency in the call routing process are reduced, providing for increased 
call ihrouuhpui. 

Addiiionally, the particular means for transmitting the calls from the modem to 
20 the i:.iieua\ is not particularly important. An RS 232 cable, a packet bus, or the 
inicmai bus of a host personal computer (such as the ISA, EISA, PCI or a VESA bus) 
may be used. While the above examples refer to SLIP and PPP processing in the 
modem computing platform (or other applicable platform such ISDN TA or CSU), the 
invention is apphcable to other higher level protocols. 

25 

15 

_9926387A1_IA> 



wo 99/26387 PCT/US98/24290 

REPRESENTATIVE NETWORK ACCESS SERVER 
IMPLEMENTATION 

FOR PPP OR SLIP DISTRIBUTED PROCESSING 

? The present invention may also be implemented in a communication 

processing system that is depicted generally in FIG. 2. A plurality of call originators 
20, 22 and 24 are located at various remote locations, which transmit incoming 
communications to a network access server 30. Call originator 20, 22 and 24 may 
consist of a personal computers CI, C2 and C3, respectively, that generates digital 

10 data and transmits the data to a modems Ml, M2, and M3, which modulates the data 
onto a telephone lines 40, 42 and 44, respectively. For the purpose of this 
specification, the particular type of call originators is not important, and the call 
originators of FIG.2 are chosen for purposes of illustration only. The call originators 
have communication software that uses the PPP or SLIP protocol. 

15 In FIG. 2, the data that is transmitted onto the telephone lines at 40, 42 and 44 

is in analog form. The illustration in FIG. 2 assumes that the communication system 
makes use of the public switched telephone network (PSTN) 50 such as the Tl 
network. The calls from the call originators are digitized and placed into one of the 24 
multiplexed channels of the four-wire Tl span line 51 by the telephone company and 
20 fed into the network access server 30. As used herein, the term Tl span line refers to 
twenty-four 64 kbps (thousand bit per second) DSO channels that are multiplexed in 
the 1.5444 Mbps DSl rate, with each DSO charmel carrying the digital representation 
of an analog voice channel. The term "trunk," as used herein, refers to a single DSO 
channel. 

25 The digital signals representing the incoming communications are fed into the 

network access server 30 by the Tl span line 51 (or possibly two span lines). The 
network access server 30 then routes the incoming call onto the network 52. The 
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network may be a Token ring network, Ethernet, Internet, or other type of network, 
the particular details of which are not important. The host computer system 60 then 
receives the call and process the calls as needed. The host computer system 60 
depicted in FIG. 2 consists of a variety of computers such as a personal computer C5, 
5 data storage terminal C4, and mainframe computer C3. As was the case with the call 
originators 20, 22 and 24, the details of the host computer system 60 has the capability 
of sending out calls via the network access server 30 to the remote data terminals 
(such as call originator 20). 

Referring now to FIG. 3, the network access server 30 is shown in a functional 

10 block diagram form and will be described in more detail. A description of the 
network access server is set forth in the patent of Dale Walsh et aL, U.S. No. 
5,525,595, and the reader is directed to the '595 patent for a more exhaustive 
description. The network access server 30 has a chassis 70 which houses a telephone 
interface unit 72 which receives the incoming calls on Tl span line 51, demultiplexes 

15 the calls, and routes the calls over a high speed time division multiplexed (TDM) bus 
complex 74 to twelve quad modem modules 76A, 76B, etc. Each modem module 76 
has four modems (not shown in FIG. 3) which demodulate the incoming calls. Thus, 
if there are two Tl span lines 51 incoming to the network access server 30, there are 
48 modems in all for the 48 DSO channels incoming into the network access server 30. 

20 The connections on the TDM bus complex between the telephone interface unit and 
the modems are static or "nailed up" connections, and are established on power-up of 
the network access server 30. The TDM bus complex 74 carries data back and forth 
between all of the various modules of the network access server 30. A high speed 
parallel bus is also part of bus complex 74 and transmits data and messages in packet 
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form between the modem modules after demodulation and the network gateway 
module 82. 

Each modem module 76 is provided with a corresponding modem network 
interface module 78 A, 78B, etc. The modem network interface modules 78 have four 
s\ nc/async RS 232 ports 80. The RS 232 ports 80 are linked to computers of the host 
computer system and may be used to output the calls from the network access server 
3n to liie host computer. 

The network access server 30 also includes a gateway application module 82, 
uhich functions as a routing and processing engine for directing calls from the 
uctuiuk access server to the local area network and vice versa. A representative 
•J jtc\v j\ application module is described in the above-referenced patent to Dale Walsh 
c: al , I S No. 5,528,595, and the reader is directed to the Walsh et al. patent for a 
Tuorc detailed discussion. Gateway cards for use in a network access server are 
cmnnicrcijily available, such as the NetServer ™ and EdgeServer'^'*^ cards from 3Com 

1:^ ( \>rpor.Mum, and such devices are available from other manufacturers of network 
access Ncrvcrs. A network management module 86 provides management and 
>up.T\ I MOM functions for the network access server 30. The reader is directed to the 
paicrn oi Rozman et al., U.S. No. 5,438,614, which is incorporated by reference 
herein, for a more detailed discussion of a modem management system for use in a 

20 network access scr\'er. 

The telephone interface unit 72 of Figure 3 is described in detail in the Walsh 

cl al. '5*>5 patent, therefore the reader is directed to the patent for a detailed discussion 

of ii.s Ciinsiruciion and functionality. The card is composed of two separate modules, 

an incomini: call interface 105 module and an incoming call application 175 module. 

25 The interface module 105 physically receives the incoming Tl span lines, converts the 

18 
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signal in a digital TTL format, and delivers the signal to the network application 
module 175. The interface module 105 provides a CSU interface which recovers 

\ clock signals and data from the incoming Tl signals, and also provides the 

transmission of outgoing digital telephone signals representing digital data to line Tl 
5 The application module 175 provides framing of recovered Tl data to extract the Tl 
DSO channel data and then switches the channel data twenty four time slots on a TDM 
bus in complex 74 for distribution to the all-digital modems in the modem modules 

I 76. 

The digital modem modules 76 and 78 are also described in the Walsh et al. 
10 '595 patent in detail, therefore the reader is directed to the Walsh et al. '595 patent for 
a detailed discussion of the modem cards, their architecture and function, and their 
interfaces to the TDM bus and the parallel bus on complex 74. The cards are available 
commercially from 3Com Corporation, and similar modem cards are available from 
other manufacturers in the industry. Each module 76 contains four modems for a total 

■ 1 5 of 24 modems for 6 modem cards. As a result, the network access server 30 of FIG. 3 

can handle a total of 24 simultaneous full duplex channels of data communication. If 
two Tl span lines are inputted into telephone interface unit 72 (FIG. 3) then 12 

I modem modules may be provided to handle 48 simultaneous full duplex channels. Of 

course, additional capacity may be provided if desired. 
20 The protocol processing for the incoming and outgoing calls, in the present 

embodiment of the invention, is distributed among four modem control processors, 
one per modem, in the card 76. The data processing and modem control processing 

^ functions of the DSP data pump and the modem control processor may be combined 

into a single digital signal processor. Additionally, part of the protocol processing is 
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performed in the computing platform in the gateway module 82, as described in detail 
below. 

Distributed Processine of PPP and SLIP protocols 
5 With the above description in mind, the reader's attention is directed to FIG. 4 

and FIG. 5, which is a flow chart of the distributed PPP and SLIP processing 
procedure according to one form of the present invention. The following discussion is 
made in reference to the network access server ernbodiment of the invention, and 
persons of ordinary skill in the art will readily understand that the description can be 

! ' ' aviapicd lo other possible embodiments. 

At step 200, a call is arrived from one of the remote call originators, or else a 
call is initiated from the host computer system to a remote computer or other data 
terminal. A step 202, the call is answered in a well known manner. At step 204, the 
gateway application module 82 (FIG. 3) determines the high level protocol of the 

15 incoming or outgoing call. This is done by determining, for example, the port 
configuration of the call, by the telephone number of the call originator or the call 
destination, by conversation or menu with the user, by a user name and/or a database 
look-up, by an automatic detect or inspection of data routine, or whatever other 
process is implemented. 

20 At step 206 the gateway module 82 configures the protocol mode for the 

modem. The software routine CONFIG. l.S set forth in published PCT application 

serial no. PCT/US96/09661 of Daniel L. Schoo et al. is implemented in this step. The 

above PCT application is incorporated fully by reference herein, and references in this 

document to ''software listing" refer to the software listing appended to the above PCT 

25 application. If the asynchronous transparent protocol is performed, at step 214 a 

20 
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forward on timeout or buffer full command is sent to the modem in the modems 
module 76. If the SLIP protocol is used, for receiving calls at step 210 the modems 
remove transparency and translation characters, and send the payload (a number of 
bytes) or overflow errors to the gateway module 82. In the above-referenced software 
5 listing, routines in SLIPRX.S are implemented at this step. For transmitting calls, at 
step 212 the modems add transparency translation characters and a SLIP framing 
character. In the appended software listing, the routines in SLIPTX.S are 
implemented. 

If PPP protocol is used, the routine in CONFIG. l.S is implemented at step 
10 208. Step 208 of FIG. 4 is shown in detail FIG. 5. Referring now to FIG. 5, at step 
2 1 6 the gateway module 82 enters the LCP negotiation procedure per the instructions 
set forth in the Request For Comments (RFC ) 1661 Standard. This is a publicly 
available document which specifies an Internet standards track protocol for the 
Internet community. Persons of ordinary skill in the art are familiar with the RFC 
15 1661 document. 

At step 218, the gateway module configures the modems for each LCP 
negotiated parameter. This may be accomplished in order, for example, MTU, TX 
Async map, RX Async map. In the appended software listings, the routine in 
CONFIG2.S describe this procedure. The procedure then enters a data transfer phase 
20 at step 220 in which ^'messages" are uploaded or downloaded through the network 
access server 30. In the receiving or uploading direction, the modems remove the 
HDLC frame characters, and send the payload, number of bytes and error code (such 
as good FCS, bad FCS, overrun, or partial). This procedure is set forth in the software 
listing PPPRX.S. 

21 
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In the transmitting (downloading) direction at step 224, the modems get the 
payload from the gateway module 82, perform character transparency translation and 
transmit asynchronous map, calculate the FCS, and add HDLC framing characters. At 
step 226, the messages are sent out through the telephone network 5 1 . This procedure 
5 is set forth in the PPPTX.S routine in the software listing. 

One representative example illustrating the advantages of the invention in 
terms of throughput over the prior art methods can be seen if FIGS. 6-9. FIG. 6 is a 
) graph of the upload throughput using prior art PPP protocol processing in a single 

platform at the network gateway. Note that the total throughput of the network access 
' 10 server remains substantially constant after a peak is reached with 8 active ports and 

levels out at 1 7 active ports at 24,000 bytes per second. The per port throughput starts 
off quite high at roughly 4600 bytes per second for a small number of active ports 
I (that is, less than 10), but drops dramatically at higher numbers of active ports. Thus, 

^ FIG. 6 illustrates the processing "bottle-neck" that occurs when the PPP protocol 

15 processing is performed in a single platform at the network interface. 

FIG. 7 is graph of the upload throughput when the distributed processing 
technique according to the teaching of the present invention is used. Note that in FIG. 

I 1 the total throughput rises steadily up to 18 ports, dips slightly between 18 and 21 

I 

\ ports, and then increases up to total throughput of approximately 92,000 bytes per 

20 second with 25 active ports. Between 25 and 48 active ports, the total throughput 

remains between roughly 72,000 bytes per second and 86,000 bytes per second. This 

is approximately 3 to 4 times the total throughput as compared to the results of FIG. 

6. Further improvements in system tuning would smooth out the oscillations in 

system throughput shown in FIG. 7, especially for higher numbers of active ports. 

25 Note also that the average per port throughput using the techniques of the present 
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invention remains relatively high as the number of active ports increases, 
accomplishing an efficiency improvement of more than 3 times compared to the 
results of FIG. 6, particularly at higher volumes of active ports. 

FIG. 8 is a graph of the download throughput when the PPP processing is 
performed in the prior art technique of a single platform at the network gateway. In 
FIG. 8, note that the total download throughput levels off at approximately 55,00 
bytes per second between above 20 active ports when the prior art techniques are used. 
The average per port throughput starts at roughly 45,000 bytes per second with under 
10 active pons, but this value drops steadily to a low of approximately 12,000 bytes 
per second when 48 active ports are used. 

FIG. 9 is a graph of the download throughput when the distributed processing 
technique according to the teachings of the present invention is used. Note that the 
total download throughput rises steadily to a high of approximately 104,000 bytes per 
second when 26 active ports are used and remains relatively constant at higher levels 
of call activity. As can be seen by comparison of FIG. 9 to FIG. 8, this is a 
substantial improvement in call throughput. The average per port throughput using 
the techniques of the invention remains relatively constant at 4500 bytes per second 
up to 22 pons and this value drops gradually to approximately 2000 bytes per second 
when 48 pons are used. This represents an improvement of approximately 2 to 1 over 
the download per port throughput, particularly, at high call volumes. 

. Presently preferred software routines for implementing the procedures shown 
in FIG. 4 and FIG. 5 are set forth in code form in the Appendix to the above- 
referenced published PCT application. A person of ordinary skill in the art will 
appreciate that the code represents but one example of the implementation of the 

invention in a network access server environment. The code is stored and executed in 
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the modem DSPs of the modem modules 76 of FIG. 3. The software routines in the 
gateway module 82 that interact with the modem software routines comprise 
straightforward higher level control routines, with which can be readily developed by 
persons of ordinary skill in the art. 

DISTRIBUTED PROCESSING OF RTF AND RTCP PROTOCOLS IN 
HIGH DENSITY NETWORK ACCESS SERVER 

An alternative embodiment of the invention, involving distributed processing 
of RTF and RTCP, will be described below in conjunction with a high density 
version of a network access server. It will of course be readily apparent that the 
distributed processing of RTP and RTCP can be performed in the network access 
server of FIG. 3, and the PPP and SLIP distributed processing can also be performed 
in the embodiment described below. 

The architecture of the alternative network access server 30 embodiment is 
shown in FIG. 10. The embodiment of FIG. 10 is a high density network access 
server, in that each general purpose DSP or modem card 376A, 376B, etc. contains a 
high density DSP configuration capable of handling 23, 24 or 30 DSO channels, 
instead of four DSO channels per modem card in the embodiment of FIG. 3. By 
providing a set of high density modem cards 376 and a robust computing platform in 
the network gateway 382, a single chassis can process many hundreds of calls through 
the device simultaneously, as compared to 24 or 48 calls in the embodiment of FIG. 3. 
The term "HDM" for the DSP cards 376 A, 376B etc. in Figure 10 is an acronym for 
''high density modem," indicating that each card performs modem functions for a 
large number of channels on the telephone line, such as 23 B channels plus 1 D 
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channel for an ISDN Primary Rate Interface, 24 DSO channels for a TI line and 30 
channels for an EI line. 

In the embodiment of FIG. 10, each DSP card 376 has its own Tl/ISDN 
telephone line interface 372 connected to an ISDN PRI or TI line. The Tl/ISDN 

^ telephone line interface 372 is similar in architecture and function of the interface 72 
i>r FIG. 3 and set forth in the Walsh et al. '595 patent, and is connected to the high 
density modem cards by a TDM bus, as described in detail in the Walsh et al. '595 
patent An alternative would be to provide a plurality of TI/ISDN telephone line 
KUeri.ices "2 and distribute DSO channel data to the modems via a TDM bus with 
cxtr.i highway lines, as described above in the embodiment of FIG. 3. 

Tlie hiL!h density DSP cards 376 are cormected to the gateway card 382 via a 
hi-^h speed parallel packet bus 374, similar to that described in the Walsh et al. patent. 
Ihc ru:niher ot" high density DSP cards 376 and associated telephone line interface 
eard . / "J is essentially arbitrary, but 10 to 24 such cards are typical in a high density 

5 nct\\Mrk access server application today, providing modem functionality for between 
r-if ' aiu! 5 ' T 1 DSO channels. The HDM cards are described in detail below. 

I he eaieway or EdgeSer\'er "^^^ card 382 consists of a general purpose 
compuime piaiform (such as an IBM PC) running a stand alone or shareware network 
operaimi: s\ stem such as Windows NT ™ from Microsoft Corporation or UNIX. The 

0 cani 7vS2 contains software and hardware modules to perform call routing, modem 

coniieuration and other features as set forth and described for the gateway modules in 

the Walsh ci al. '595 patent and the Baum et al. patent, U.S. No. 5,577,105, also 

incon>orated by reference herein. Further details on the design and features of the 

EdLieServ er^'^* card 382 are set forth in the patent application of William Verthein et 

5 al. Serial No. OS S 1 3. 1 73, the contents of which are incorporated by reference herein. 
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The network access server 30 shown in FIG. 10 is useful for a number of 
different types of applications, such as Internet access, remote access to corporate 
i backbone networks, video and audio conferencing, Internet telephony, digital wireless 

Internet and corporate network access, to name a few. In an Internet telephony 
5 embodiment, the product provides a facility for users to engage in long distance 
telephone, audio/visual and/or data sessions using the Internet as the transport medium 
rather than the long distance public switched telephone network of the inter exchange 
I carriers. Users realize a substantial savings in transmission charges as compared to 

phone charges. 

10 . A standards-track protocol has been develop by the industry to provide end-to- 

end delivery for data with real-time characteristics, such as audio, video, and 

t 

J simulative data, over multicast or unicast network services. This protocol is described 

; in detail in a publicly available document known as RFC 1889, January 1996, the 

i entire contents of which are incorporated by reference herein. In the RTP, the data 

15 transport is augmented by a control protocol referred to as RTCP (Real-time Transport 

Control Protocol) to allow for monitoring of the data delivery in a manner scalable to 

large multicast networks, and to provide control and identification functionality. The 

I RTP and RTCP are designed to be independent of the underlying transport and 

; network layers in the OSI model. In one representative embodiment of the invention 

^ 20 described below, the processing of the RTP and RTCP protocols for each call passing 

through the network access server 30 is distributed between the gateway or 

EdgeServer card 382 and the DSP platform in the HDM cards 376. 

The RTP has header fields that are either fixed or deterministically varying, 

with a certain format described in the RFC 1889 document. These fields include 

25 fields for a sequence number, timestamp, synchronization source identifiers, and 
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contributing source identifiers. The header can be extended with RTP header 
extension to provide new payload-format-independent functions that require 
additional information to be carried in the RTP data packet header. Additionally, the 
RTCP packets have a packet format and header fields for control information. 
5 The network access server 30 of FIG. 10 is an ideal platform in which to direct 

calls between remote and network endpoints of a call using RTP, The software and 
hardware functionality of the HDM DSP platform and the EdgeSer\'er platform is 
distributed in the manner shown in Figure 11. The telephone line interface cards 372 
of FIG. 10 perform PSTN access functionality, including Tl signaHng, call 

10 supervision (E&M, Loop Start, Ground Start), time slot routing and Tl framing, ISDN 
signaling and B-channel routing. The HDM cards 376 perform information flow 
processing, including tagging and delivery of audio and control components, a portion 
I of the processing of the RTP (described below), audio transcoding in accordance with 

the G.771 standard (or G.723 or G.729), and other audio processing functions (e.g., 

15 DTMF generation, echo cancellation, etc.). The EdgeServer 382 or gateway platform 
performs call management, a portion of the RTP processing (described below), H.245 
processing. H.225 processing, Internet Protocol (IP) routing, and gateway to HDM 
I coordination protocol processing. 

The arrows above the blocks in FIG. 1 1 indicate specific data forms for an 

20 Internet telephony session that are directed between the telephone line interface card 

372, HDM DSP card 376, and EdgeServer card 382 for both inbound and outbound 

data. For data going between the packet switched computer network and the PSTN 

(referred to herein as outbound data), asynchronous H.323 over IP data is received at 

the EdgeServer card 382. Audio ft-ames/Protocol Data Units (PDUs) and modem 

25 control information is passed in packet form from the EdgeServer card to the high 
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density modem card via the packet bus 374 (FIG. 10). The HDM DSP platform in the 
card 376 performs additional RTP processing, and other audio processing described 
herein, including jitter buffering, G.71 1, G.723 and G.729 transcoding, and Tl/ISDN 
siunahng. The data is sent in PCM (pulse code modulated) form from the HDM card 
lo ihc telephone line interface 372 over the TDM bus (Figure 10), framed into Tl or 
ISDN PRI frames, and sent on the PSTN for transmission to the destination. 

FIG. 12 is an overview illustration of a presently preferred distributed RTP 
i pri>ccssing architecture for the network access server embodiment of FIG. 10. 

Nutirccs of audio/video or other real time data, such as a plurality of H.323 personal 
V i»:i!puicrs, arc located on the packet .switched LAN or WAN connected to the 
nctvvt>rk access server. The data from the H.323 PCs comes into the EdgeServer cards 
.IN I. W irartlc (over IP). 

^ in one embodiment, the EdgeServer card 382 handles all RTP ftinctionality, 

? \v nil exception of jitter buffer and all frame-based audio processing and generation 

■ 1^ RTCF* send and receive reports, which are performed in the HDM card 376 

pi.it lonns The actual processing performed by the EdgeServer 382 includes the 
h.i:uiiMJL; o! tlic creation and destruction of the audio streams as directed by the H.245 
^ and 11 225 protocol. Additionally, the gateway or EdgeServer card 382 acts as a 

i] ccnirai ciiordinator for the RTP streams and provides the HDMs with all the 

20 inioni^aiion thai lliey need to provide their part of distributed RTP functionality. (In 
one pD.ssihIc variation, the gateway card 382 also terminates all RTCP traffic, 
t inchidmu termination and generation of sender reports and receive reports. IP, UDP 

* and RTP headers are placed onto all audio frames sent to the LAN. The EdgeServer 

3S2 tills in all RTP header fields that change on a dynamic basis.) The EdgeServer 
25 also prox ides a translation function for software interfaces between the EdgeServer 
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card 382 and the LAN interface, and between the card 382 and the packet bus 374. 
hi one possible variation, the RTCP processing is distributed between the computing 
platform in the card 382 and the HDM card 376 platform, such that information for 
the send and receive reports for RTCP comes from the HDM platform. The actual 
send and receive reports may be sent out by either the HDM modem platform in the 
cards 376 or by the computing platform in the gateway card 382. 

Still referring to FIG. 12, the HDM 376 digital signal processor computing 
plaiform handles the processing of the RTF jitter buffer and all lower level audio 
processing that occurs below this level, including DTMF generation, echo cancellation 
and transcoding. The actual processing that is encompassed within the HDM platform 
includes a coordination function with the Edgeserver card to respond to its setting up 
new RTF session, or tearing down old sessions; providing a jitter buffer to remove any 
arrival jitter; perform all RTF header encapsulation and de-encapsulation functions; 
perform all IF and UDF header encapsulation and de-encapsulation functions; and 
reacting lo changing Internet/Intranet conditions by changing the size of the jitter 
buffer. 

An embodiment of distributed processing of RTF functionality between the 

EdgeSer\xr and the HDM cards is shown in more detail in FIG. 13. The left hand 

margin shows the form in which data (such as audio data) is transmitted between the 

0 user H.323 client on the computer network and the FSTN phone client linked to the 

network access server via the FSTN. As can be seen in the Figure, audio is sent 

encapsulated in an RTF header from the H.323 client all the way into the HDM 

platform 376. The HDM platform 376 removes the RTF headers and performs jitter 

buffering in a jitter buffer, with the size of the jitter buffer dynamically changed in 

5 order to deal with the bursty, asynchronous nature of packet switched data from the 
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computer network. The modem platform 376 also performs lower level DSP 
processing of audio frames, including transcoding, echo cancellation, and DTMF 
generation. The audio data is transcoded according to the G.711 standard and sent 
over the PSTN to the phone client. 
5 Still referring to FIG. 13, the outbound data from the LAN is translated 

through a network interface software structure 302 (WinSock, BSD sockets or TDI), 
the details of which are not important and readily derived by persons of skill in the art. 
A receive RTP packet transfer module 304 operates with a Calculate RTP Receiver 
Reports module and advises the user H.323 client of the number of lost packets, size 

10 of the jitter and other information per RFC 1 889. The audio data encapsulated in RTP 
header is translated by the S-Bus interface software structure 308 (described below) 
and sent over the packet bus 374 (FIG. 10) to the HDM modem cards 376. 

A reduced instruction set central processing unit (RISC) computing platform 
in the HDM card implements a similar S-bus interface software program 310 and 

15 receives the RTP audio data, and performs the jitter buffering (described below). The 
RISC computing platform 312 calculates the needed size of the jitter buffer based on 
LAN/WAN conditions and any intra-chassis latency. The RISC computing platform 
312 removes all RTP headers and passes audio frames to the DSP computing platform 
314 in the HDM card. The DSP platform performs other lower level audio 

20 processing, including DTMF generation, echo cancellation, and ultimately the G.711, 
G.723 and G.729 transcoding. The inbound processing is as described in Figure 13, 
with the encapsulating of audio framing with RTP headers performed by module 316. 

While the embodiment of Figure 13 shows the processing of RTCP send and 
receive reports on the EdgeServer card 382, it may readily be shifted to the high 

25 density modem platform in the HDM card 376. For example, the RTCP protocol 
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processing may be distributed among the gateway platform and the DSP platform in 
the HDM card, for example the information for the RTCP send and receive reports 
comes from the modem platform but the actual send and receive report may be 
generated and sent out by either the DSP platform in the HDM card or at the gateway 
5 card 382. 

Referring now to FIG. 14, the software architecture for the S-BUS driver 
interface for the EdgeServer card 382 is illustrated in block diagram form. The 
interface consists of four driver layers dynamically linked together: appUcation 
drivers (system control, Q.931 messaging. Voice Prompt, H.323 interworking, and 

10 Real-time Audio); a message router driver 352, an S-BUS messaging driver 354 and a 
packet bus (PB)- PCI driver 356. 

All application drivers 350 receive and transmit their message via the message 
router driver. The message router driver is the holding place for various message 
types such as audio, data and control, which are queued into three corresponding 

15 transmit or receive priority queues. The system control driver interfaces with the 
message router driver to send requests and receive events to and from the SBUS 
message driver 354. The system control driver acts as a module-to-module 
communication for the underlying EdgeServer components. The module-to-module 
communications include the following types of messages: 

20 • Call establishments - Call establishment messages are used to signal the 

HDM or System Control of various call establishment requests and 
responses. For example. Dial-out, Answer Call, Disconnect Call, 
Connection Failed/Succeeded, and Incoming Call. 
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• Call maintenance - Call maintenance messages are used by the System 
Control to request RDM's statistics in real-time. Likewise, the HDM 
responds back to the System Control with the appropriate message. 



5 • Resource management - Resource management messages are used by the 

System Control to request various HDM's resource information. For 
example, availability of HDMs and its number of ports/channels, DSPs 
I utilization, and its number of DSPs available for transcoding. Likewise, 

the HDM responds back with the appropriate resource information. 
H» • Configuration - Configuration messages are used by the System Control to 

send configuration messages to an HDM to add and/or modify the flavor of 
a connection. For example, Enabling/Disabling HDM Ports. 

• Diagnostic - Diagnostic messages are used by the System Control to 
1 5 request a specific test to be executed at the HDM level in conjunction with 

the associated EdgeServer's test code. For example, various level of 
ioopbacks at the HDM's hardware components, client's terminal loopback, 
I and measuring round-trip delay. 

20 The Q.931 Driver 358 is responsible for sending Q.931 messages to HDM and 

processing receive Q.931 messages from the HDM. Q.931 is a message-oriented 
control protocol for call setup/tear-down in the ISDN environment. All Q.931 
messages are control messages which will be processed by this driver and the HDM. 
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Voice Prompt driver 360 is an optional driver and manages the record, store 
and play back of voice prompts. 

The H.323 Interworking Driver 362 interfaces to the Message Router Driver to 
send requests and receive events to and from the SBUS Message Driver. The H.323 
5 Interworking Driver relationship with the SBUS drivers is to communication the 
following message type between the EdgeServer and HDM: 

• Audio Mode Exchange with HDM. 

• Open an Audio path between SBUS drivers and Real-time Audio Transfer 
Driver. 

10 All H.323 interworking messages are control messages which will be processed 

by this driver and the HDM or SBUS drivers. The Real-time Audio Driver 364 sends 
and receives packetized audio via SBUS Messaging driver. Audio packets are 
enqueued and dequeued in the audio queue of the Message Router 352. The Message 
Router Driver 352 is an intermediate driver in kernel-mode which connects to the 

15 Application drivers above it and the SBUS Messaging Driver 354 below it. The 
Message Router 352 provides the interface to support the various multimedia message 
types (audio, data, and control) that are communicated between the Application 
drivers on the EdgeServer and the HDMs. Each multimedia message that is received 
from the HDM, the SBUS Messaging component parses the packet's multimedia 

20 header to distinguish its priority level and pass this message to the appropriate 
Message Router's priority queue. From this queue this driver passes the multimedia 
message to the appropriate Application Driver 350 based on the application ID in the 
multimedia header. Likewise, the Message Router 352 transmits the multimedia 
messages from the Application Drivers 350 to the SBUS Messaging's 354 transmit 
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priority queues. The Message Router 354 communicates to the HDM using the 
hardware and data link services of the SBUS drivers. 

The PB-PCI driver component 356 is a Windows NT^^" driver which interfaces 
with the packet bus hardware, for the purpose of data delivery between the 
5 EdgeServer, HDM and other cards on the network access server 30. The SBUS 
Messaging component utilizes the PB-PCI driver as a physical level device to transmit 
and receive messages between the EdgeServer and HDM cards. 

FIG. 15 is a high level block diagram of the high density modem cards 376. A 
TDM interface 380 provides an interface to a TDM bus leading to the Tl/El interface 
10 card (see Figure 10). The details of the TDM bus are known in the art. and the reader 
is directed to the similar interface in the Walsh et al. patent for the quad modem cards. 
The DSO channel data is sent from the TDM interface 380 to a DSP subsystem 382, 
comprising 12 high performance Texas Instruments TMS3201c548 digital signal 
processors (DSPs) 384, each DSP supporting bidirectional modem functionality for 
15 two separate DSO channels. The DSP subsystem 382 is connected by a bus system 
386 to an IBM PPC403GCx reduced instruction set central processing unit (RISC) 
388 functioning as a board manager, and to a shared memory 390. The shared 
memory 390 is connected to two additional IBM PPC403GCx RISC application co- 
processors 390, and to a packet bus interface 394. The RISC co-processors 392 
20 perform the functions illustrated in FIG. 13. The hardware details of the packet bus 
interface 394 are conventional and described in the Walsh et al. patent. 

The following summarizes some of the major functions for the network access 
server performed by the high density modem card 376: 
• ISDN Signaling 
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• PSTN (Channelized T 1 ) Signaling 

• Resource Management 

• Data Path Setup and Thread Management 

• Flow Control 

5 • Packet Bus/S-Bus Processing 

• RTP/UDP/IP Header Processing 

• Jitter Estimation and Dejittering 
\ • Packet Sequence Re-ordering 

• Audio Packetization 
i 10 • Audio Transcoding 

■i 

• Echo Cancellation 

: • Automatic Gain Control 

" • DTMF Tone Generation 

Most of the above functions are conventional and well known in the art. The 
15 functions relating to RTP processing are described in detail below. 

In the illustrated embodiment, one HDM card support 24 calls (i.e., a Tl span). 
Therefore the HDM architecture is based on the HDM 24/30 hardware architecture 

ij 

\ consisting of 12 high performance Texas Instruments TMS3201c548 DSPs and 3 IBM 

PPC403GCx RISC processors. An alternative embodiment for a 30 channel carrier, 
20 such as EK would include a daughter board which supports four additional C548 
DSPs, thereby providing some redundancy in the DSP engines. 

The HDM is an evolution of the digital modem technologies found in the 
Walsh ct al. patent, providing a highly integrated processing solution for purely digital 
configurations of a network access server. The HDM does not support RS-232 and 
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POTS interfaces. The existing HDM 24/30 will terminate both analog modulations, 
via A-law or z^-law PCM encoded data, and ISDN calls such as V.120, X.75, or Async 
to Sync PPP at 64Kbps/56Kbps or V.l 10 from 300bps to 38.4Kbps. 

As noted above, the HDM 24/30 NAC is designed to interface to the telephone 
5 line interface card that provides a direct interface to either a Tl or El span. There are 
two types of Tl signaling functions supported in the HDM: channelized Tl and PRI. 
For channelized Tl, the HDM supports 24 DSO channels; for PRI, the HDM supports 
I 23 B-channcls and 1 D-channel; for El the HDM supports 30 B channels and 1 D- 

channel. 

iO FIG. 16 is a block diagram of the software architecture of the HDM 376. The 

; software is designed to support both call control processing and data processing. A 

• packet bus driver module 400 interfaces with the packet bus 374 (see FIG. 10) to 

receive and transmit packets of data between the HDM and the gateway or EdgeServer 
card. A Packet bus S-BUS messaging routine resides above the packet bus driver and 
15 performs translation of S-BUS message fields from the EdgeServer. Control and 
signaling data is sent to the Board Manager call control module. Signaling and call 
control for telephony and ISDN calls are separated and sent to a signaling and control 

I. manager 406, which supervises a DTMF or other tone generator 408, a Tl signaling- 

4 

call supervision manager 410, and an ISDN Q.921/Q.931 signaling manager 412. 

20 The signaling and call control data is sent to a PCM signal interpreter/transcoding 

module for required signal conversion into DSO channel data. 

In the HDM, audio data is passed from the Packet Bus S-BUS messaging 

module 402 to a dynamic jitter processing module 420 for removing jitter in the audio 

stream. A board manager data path module 422 supervises the jitter processing, RTP 

25 header processing, and audio packetization and depackatization ftinctions, performed 
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by modules 420, 424 and 426, respectively. An audio transcoder module 428 resides 
below the audio packetizing and depacketizing module for performing G.711 
transcoding. Below the audio transcoder, an echo cancellation module 430 and an 
auioniatic gain control (AGC) module 432 perform lower level processing. The 
audio data is passed to the PCM signal interpreter module 414 and forwarded to the 
Ti ISDN telephone line interface 372 (FIG. 10) as DSO data for framing and 
transmission on the telephone Une to the destination. 

I The call control processing performed by the HDM is shown in FIG. 17. The 

•ioUiCX RISC chip platform 388, 392A, B (FIG. 15) performs the required S- 
1' iU S f'ackci Bus processing 400, 402 for inbound and outbound data between the 

j fIDM a!ul the EdgeServer, the board manager call control module 404, the 

tclcriu>n> ISDN signahng and control processing as indicated in the Figure. The 

I DSI* platii)nn performs the DTMS/MF tone generation and call signaling tone 

I dL-uxtion 43S. The DSP platform also implements the PCM signal interpreter module 

; 15 414 

Kctcmng to FIG. 1 8, the HDM data processing for the HDM is shown in block 
diagram ibnn. As can be seen, the distribution of functionality between the RISC 
i processing and the DSP processing is slightly different for inbound and outbound 

' data \ oT inbound data, the DSP performs processing required at the TDM interface, 

20 A( ii \ cciu) cancellation and audio transcoding. Audio packetization and the addition 
ot RJV I DP IP headers for the data is performed by the RISC chips, as is the S-BUS 
and !*ackct Bus processing. 

For outbound data, after the S-BUS and Packet Bus processing by the RISC 
chips, ihc DSP assumes the task of performing jitter processing, removing 

25 RTP/UDP/IP headers, and audio depacketization, and the lower functions of audio 
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transcoding, echo cancellation and TDM interface processing to produce DSO channel 
data for the respective Tl /ISDN network interface card, 
i The major resource in the HDM required for supporting internet telephony in a 

network access server is the DSP transcoding resource. Based on the system 
5 requirements, it is possible to deliver 3 full duplex G.723/G.711 transcoders in each 
Texas Instruments C548 DSP engine. As a minimum, each C548 DSP (12 per HDM 
card) is required to handle two full-duplex audio transcoding tasks. Figure 19A 
1 illustrates a HDM/DSP subsystem configuration for such a minimum configuration. 

To suppon an embodiment providing both remote access services and Internet 
10 telephony applications, a more flexible and extensible architecture is required. A new 
DSP architecture can be designed with the goal of creating an extensible architecture 
which can be migrated to support additional applications and features without 
i significant modifications to the architecture. In such a system, an event-driven DSP 

:i architecture that is flexible and robust is preferred, but which fixes the functions that 

15 will be supported at the compile time. The DSP resources might be allocated to V.34 
tasks, audio CODEC tasks, or any combination that is allowed by the available 
resources. A proposed event-driven architecture that allows each DSP to perform a 

^ mixture type of tasks with the goal to fully utilize the processing power of each DSP 

i ■ , 

i engine is shown in FIG. 19B. Ideally, in such an architecture, the goal is to produce 

20 an architecture with an operating system that allows multiple functions to run and 
modularize functionality so that one can modify /add/delete modules with very* little 
modification to the operating system. 

RTP/UDP/IP Header Processing Module 424 
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The RTP/UDP/IP header processing is supported by the apphcation processing 
layer in the RISC. The RTP/UDP/IP header processing module is responsible for 
adding the 12 octets of RTP header, 12 octets of UDP header, and 20 octets of IP 
header to all the audio packets which are transmitted from the PSTN to the LAN. 
This module is also responsible for removing the RTP/UDP/IP header from the audio 
packets for data from LAN to the PSTN. The RISC application co-processors also 
perform PPP/SLIP framing, as well as the RTP/UDP/IP processing. 



G.723.1/G.71 ] Audio Transcoding Module 428 
10 The audio transcoding between G.723.I and G.711 is required for the DSP 

subsystem since the G.711 audio stream will always arrive from the PSTN/ISDN 
clients over Tl and the G.723 compressed stream arrive from the LAN side. 

Specifically, the following functions are supported by the DSP subsystem: 

• Each DSP engine supports 2 concurrent full-duplex G.723. 1/G.71 1 transcoder 
15 tasks. 

• The transcoder implementation is compliant to the G.723. 1 and G-711 ITU 
implementation. 

• It supports encoder/decoder independence such that one can allocate any 
combination of encoders or decoders according to system configuration and within 

20 the DSP resource limits. 

• It supports synchronous output to the Tl interface. 



Jitter Processing Module 420 and Packet Re-Ordering Module 426 
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Jitter (as defined previously) can be caused by network congestion external to 
; the network access server 30, or the network access server's internal communication 

; and processing. The network congestion may also cause the packets to arrive in a 

wrong order, a phenomenon termed a packet sequence disorder. These conditions are 
amehorated by the jitter processing module 420 and the packet re-ordering module 
: 426. 

The DSP subsystem 382 supports a dynamic jitter processing module 420 
here tiic size of the jitter queue is adjusted during the run time operation. This 
nuKiiilL- 420 will also support the re-ordering of the frame sequence. In addition, the 
] !" iiuHiuic will support the talk-spurts processing to reduce the bandwidth usage in the 

; LAN 

The RISC application co-processors 392 support an audio packetization 
I lunction 42(). This audio packetization function conform to the ITU H.225 Annex F - 

^ new audio packetization for G.723.1. Both 6.3kbps and 5.3 kbps rates are mandatory 

15 pan oi the G. 723.1 encoder and decoder. A G.723.1 frame can be one of three sizes: 
24 h\ic>. 2i} bytes, or 4 bytes. These 4-byte frames are called SID (silence insertion 
dcscTipron and are used to specify comfort noise parameters. There is no restriction 
I i>n hov\ 4. 20. and 24 bytes are intermixed. The first two bits in the frame determine 

\ the tranic boundary. It is possible to switch between the two rates at any 30 ms frame 

20 bi>undar\ Tins packetization scheme is compliant to RFC 1890 for the packetization 
inicr\ al with the following specification: 

I. The first packet of a talk-spurt (first packet after a silence period) is 
tiisiinijuished by setting the market bit in the RTP data header. 

2. The sampling frequency (RTP clock frequency) is 8000 Hz. 
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3. The packetization interval should have a duration of 30 ms (one frame) as 
opposed to the default packetization of 20 ms 

4. Codecs should be able to encode and decode several consecutive frames 
within a single packet. 

5. A receiver should accept packets representing between 0 and 180 ms of 
audio data as opposed to the default of 0 and 200 ms. 



Echo cancellation Module 430 

The DSP subsystem 382 supports a line echo cancellation (LEC) module 430 
\\hich complies the performance requirements of ITU G. 165 except portion of tone 
disablcr. The tone disabler defined by G.165 is primarily used to guard tones in 
telephone networks and can be implemented in the system when it is necessary. The 
echo cancellation uses signal correlation techniques to determine parameters of a filter 
that processes the incoming signal on the 4-wire side of a hybrid. The filter forms an 
15 estimate of the echo when an incoming signal is present. This estimate is subtracted 
trom the signal on the return path. 

Presently preferred embodiments have been set forth above. Persons of skill in 
the art will appreciate that modifications may be made from the disclosed 

20 embodiments without departure from the spirit and scope of the invention. For 
example, the assignment of processing to the RISC and/or DSP platforms is a matter 
of design choice and may vary from the embodiments set forth above. Further, the 
processing of the high level protocols can be distributed between the gateway and the 
modem platforms in a manner different from the described embodiments, without 

25 departure from the spirit of the invention. For example, the RTCP protocol processing 
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may be distributed among the gateway platform and the modem platform, for 
example the information for the RTCP send and receive reports comes from the 
modem platform but the actual send and receive report may be generated and sent out 
by either the modem or DSP platform or the gateway platform. As a further example, 
5 while the best mode known to the inventors for practicing the invention has been 
disclosed in the context of present or proposed commercial products of the applicants' 
assignee, it will be appreciated that the teachings are readily adaptable to other types 
of network access servers marketed by others in the industry, such as Livingston, 
Ascend, Cascade Communications, etc. This true spirit and scope of the invention is 
10 defined by the following claims, to be interpreted in light of the above description. 
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We Claim: 

1 . A method for routing data between a host connected to a computer network 
and a remote user connected to said host via a telephone line, the data associated with 
a higher level protocol for allowing the host to communicate with the remote user, the 
5 method performed in a communications access system comprising a telephone line 
interface connected to said telephone line, a plurality of modems, each modem 
associated with at least one modem computing platform, and a network interface 
having a gateway computing platform, comprising the steps of: 

receiving said data destined for said remote user and directing said data from 
10 said network interface to at least one said modems in said communications access 
; system; 

distributing the processing of said higher level protocol for said data, such that 
; said gateway computing platform performs a first portion of the processing of said 

higher level protocol and the modem computing platform in said at least one of said 
15 modems performs a second portion of the processing of said higher level protocol; and 
subsequently directing the data from said at least one modem computing 
platform through said telephone line interface onto said telephone line for 
I transmission to said remote user. 

20 2. The method of claim 1, wherein said higher level protocol comprises a real- 
time protocol associated with voice, video, or data communication. 

3. The method of claim 2, wherein said protocol comprises the Real-time Transfer 
Protocol. 

25 
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4. The method of claim 2, wherein said at least one modem computing platform 
performs the processing of jitter buffering for said data communication. 

5. The method of claim 2, wherein said at least one modem computing platform 
5 performs processing of frame-based audio data associated with said protocol. 

6. The method of claim 2, wherein said gateway computing platform performs the 
processing of creation and termination of audio streams associated with said protocol. 

The method of claim 2, wherein said gateway computing platform performs 
^ I he termination of RTCP traffic. 

S. The method of claim 1, wherein said method is performed in an Internet 

icicphony data session between said host and said remote user. 

*>. The method of claim 1, wherein said higher level protocol comprises a Point- 
To-Point Protocol. 

I 

: 10. The method of claim 1, wherein said higher level protocol comprises a Serial 

20 Line Interface Protocol. 

11. The method as claimed in any one of claims 1-10, wherein said method is 

performed in a high density network access server comprising a plurality of DSP cards 

connected by a high speed parallel bus to said network interface, each of said DSP 

25 cards providing modem functions for all DSO channels of a Tl or El span line. 
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12. In a network access server for providing connectivity between a host source of 
digital data connected to a network and a remote terminal connected to the public 
switched telephone network, comprising, in combination a telephone line interface 

5 operatively connecting said network access server to a telephone line carrying digital 
telephone signals; a plurality of DSP computing platforms for demodulating inbound 
telephone signals and modulating outbound telephone signals for transmission on said 
telephone line to said remote terminal; a bus passing digital telephone signals from 
said telephone line interface to said DSP computing platforms; and a network 
10 interface receiving data from said modems via a parallel bus and routing said data 
onto said network; 

the improvement comprising: 

said network interface comprising a computing platform processing a higher 
level network control protocol associated with said network, and wherein each of said 
15 plurality of DSP computing platforms performs the processing of a subset of functions 
associated with said higher level network control protocol associated with said 
network. 

13. The improvement of claim 12, wherein said higher level network control 
20 protocol associated with said network is selected from the group of protocols 

comprising RTP, SLIP and PPP. 

14. The improvement of claim 13, wherein said higher level network control 
protocol comprises RTP and said at least one DSP computing platform performs the 

25 processing of RTP header generation. 
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15. The improvement of claim 12, wherein said at least one modem computing 
platform performs the processing of jitter buffer associated with said RTP. 

16. The improvement of claim 12, wherein said DSP computing platforms are 
organized into a plurality of high density modem cards, each modem card associated 
with all the DSO channels of a Tl or El span line. 

17. The improvement of claim 16, wherein said higher level network control 
protocol comprises PPP. 

18. The improvement of claim 16, wherein said higher level network control 
protocol comprises SLIP. 

1 9. A method of performing the processing of RTP protocol in a communication 
session from a source connected to a computer network and a destination connected to 
a telephone line, comprising the steps of : 

passing RTP frames of data for said session from a network interface to a 
modem; 

performing jitter buffering for said RTP frames of data in a computing 
platform in said modem; 

converting said RTP frames of data into a form suitable for transmission on 
said telephone line; and 

subsequently directing said data on said telephone line to said destination. 
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20. The method of claim 19, wherein said modem platform further processes all 
frame-based audio data associated with said session. 

21. The method of claim 19, wherein said method is performed in a network 
5 access server. 

22. The method of claim 19, wherein said network access server comprises a high 
density network access server. 
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